Electronic Device with Intuitive Control Interface

ABSTRACT

An electronic device may include a display system and control circuitry. The user&#39;s environment may be presented on the display system. The environment on the display system may be a captured image of the environment, may be the actual real world viewed through an optical combiner, or may be a completely virtual image representing the environment. The control circuitry may gather information about external electronic devices in the user&#39;s environment, including determining a type and location of each external electronic device and a status of wireless communications links between external electronic devices. The display system may overlay computer-generated display elements onto the user&#39;s environment to indicate the status of wireless communications links between the external electronic devices. In response to touch or gesture input, the control circuitry may send control signals to the external electronic devices to establish or break wireless communications links between external electronic devices.

This application is a continuation of patent application Ser. No.16/056,284, filed Aug. 6, 2018, which is hereby incorporated byreference herein in its entirety.

FIELD

This relates generally to electronic devices and, more particularly, toelectronic devices that are used to communicate with other electronicdevices.

BACKGROUND

A user's surrounding environment often includes multiple electronicdevices. For example, a living room may include one or more desktopcomputers, laptops, keyboards, mice, headphones, cellular telephones,and other electronic devices. The electronic devices typicallycommunicate with one another over wired connections or over wirelesscommunications links such as Bluetooth® and WiFi® communications links.

Users may find it challenging to identify and control which devices areconnected and how the devices are connected. Typically, a user mustnavigate to a settings menu on each individual device, which thenprovides a list of external devices to which that device is connected.This process can be cumbersome and unintuitive for the user. There maybe multiple devices in the user's environment, making it challenging tomanage connections between devices.

SUMMARY

An electronic device may include a display system and control circuitry.The display system may be a head-mounted display or may be a displaythat is not head-mounted. If desired, the content on the display systemmay be virtual reality content and/or augmented reality content.

The user's environment may be presented on the display system. Theenvironment on the display may be a captured image of the environment,may be the actual real world viewed through an optical combiner, or maybe a completely virtual image representing the environment.

The control circuitry may gather information about the externalelectronic devices in the environment, including determining a producttype and location of each external electronic device and determining astatus of wireless communications links between the external electronicdevices. The external electronic devices may be presented on the displaysystem according to their respective product types and locations in theenvironment. The display system may overlay computer-generated displayelements onto the environment to indicate the status of wirelesscommunications links between the external electronic devices.

The control circuitry may send control signals to the externalelectronic devices in response to touch input on the images and/orgesture input. The control signals may be used to establish or breakwireless communications links between external electronic devices. Afinger swipe from a first external electronic device to a secondelectronic device on the display or in the environment may cause thecontrol circuitry to send wireless signals to at least one of the firstand second external electronic devices to establish a wirelesscommunications link between the first and second external electronicdevices. A computer-generated display element may be overlaid onto theenvironment in response to the wireless communications link beingestablished. The computer-generated display element may be a lineextending between the first and second external electronic devices. Afinger swipe across the computer-generated line on the touch-sensitivedisplay may cause the control circuitry to send wireless control signalsto at least one of the first and second external electronic devices tobreak the wireless communications link between the first and secondexternal electronic devices.

The control circuitry may gather information about the externalelectronic devices using input-output circuitry. For example, thecontrol circuitry may receive radio signals (e.g., ultra-widebandcommunications signals or other suitable radio signals) from an externaldevice that are used to identify the type and/or location of thatexternal device. The control circuitry may process depth informationfrom one or more depth sensors to identify the type and/or location ofan external device. The control circuitry may process camera imagesusing image recognition techniques to identify a product type of each ofexternal electronic device. The camera may capture images of a code oneach external electronic device to identify that external electronicdevice. The code may be a bar code displayed on a display, a codeemitted by an infrared light-emitting diode, a code emitted by a visiblelight-emitting diode, or a code formed from ultraviolet ink. Informationabout the external electronic devices may also be gathered by an antennathat receives radio signals from the external electronic devices and/ora speaker that receives audio signals from the external electronicdevices.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of an illustrative system in which anelectronic device with a display system communicates with externalelectronic devices in accordance with an embodiment.

FIG. 2 is a perspective view of an illustrative system in which anelectronic device presents an environment on a display system and a userprovides touch or gesture input to interact with external electronicdevices in the environment in accordance with an embodiment.

FIG. 3 is a front view of an illustrative electronic device showing howa finger swipe from a first external electronic device to a secondexternal electronic device may be used to establish a wirelesscommunications link between the first and second external electronicdevices in accordance with an embodiment.

FIG. 4 is a front view of an illustrative electronic device showing howa finger swipe across a computer-generated line extending between firstand second external electronic devices may be used to break a wirelesscommunications link between the first and second external electronicdevices in accordance with an embodiment.

FIG. 5 is a front view of an illustrative electronic device showing howcomputer-generated display elements that are overlaid onto anenvironment may have different visual characteristics to indicatedifferent types of wireless communications links in accordance with anembodiment.

FIG. 6 is a front view of an illustrative electronic device showing howtouch or gesture input on a computer-generated display element overlaidonto the environment may cause text to appear in the environment inaccordance with an embodiment.

FIG. 7 is a diagram of an illustrative system showing how touch orgesture input may be used to control an external electronic device inthe environment in accordance with an embodiment.

FIG. 8 is a perspective view of illustrative external electronic deviceswith features that may be picked up by a camera or other light sensor toidentify the external electronic devices in accordance with anembodiment.

FIG. 9 is a perspective view of an illustrative external electronicdevice with visual markers that may be picked up by a camera or otherlight sensor to identify the external electronic device in accordancewith an embodiment.

FIG. 10 is a perspective view of an illustrative external electronicdevice displaying a bar code that may be picked up by a camera or otherlight sensor to identify the external electronic device in accordancewith an embodiment.

FIG. 11 is a perspective view of illustrative external electronicdevices having a wireless communications link that is established inresponse to a finger swipe from one of the external electronic devicesto the other in accordance with an embodiment.

DETAILED DESCRIPTION

A system may include one or more electronic devices. In some scenarios,an electronic device may be used to control external electronic devices.For example, an electronic device may be used to manage connectionsbetween external electronic devices and/or may serve as an input-outputdevice for one or more external electronic devices. In other scenarios,an electronic device may send information to and/or receive informationfrom one or more external electronic devices.

An electronic device may include input-output devices that provide anintuitive way for a user to control, gather information about, orotherwise communicate with external electronic devices. The input-outputdevices may include sensors for identifying external electronic deviceswithin the user's environment, a display system for displayinginformation about the external electronic devices, and user inputcomponents such as touch sensors, force sensors, and other sensors forgathering touch input and/or gesture from a user's fingers. Theelectronic device may include communications circuitry for sendingsignals to and/or receiving signals from the external electronic devicesbased on the received user input.

For example, a camera may gather images of external electronic devicesin the user's surrounding environment, control circuitry may identifyand/or gather information about the external electronic devices, adisplay system may display the camera images in real time, and a usermay control or communicate with the external devices by providing touchinput to the display or providing other suitable user input.Computer-generated images (e.g., color-coded lines, virtual buttons,arrows, circles or other shapes, text, menus, colors, etc.) may beoverlaid onto the real-time camera images on the display to facilitateuser input operations.

In other arrangements, control circuitry may identify the types andlocations of external electronic devices in the environment (e.g., bothviewable and obstructed external electronic devices in the environment)using radio signals such as ultra-wideband communications signals,millimeter wave communications signals, or other suitable wirelesscommunications signals. The external electronic devices may be presentedon the display system according to their locations in the environment.The external electronic devices presented on the display system may becaptured images of the external electronic devices, may be actualexternal electronic devices that are viewed through an optical combiner(e.g., a head-mounted display, a head-up display, etc.) or may bevirtual, computer-generated images representing the external electronicdevices.

An illustrative system for controlling or otherwise communicating withexternal electronic devices is shown in FIG. 1. As shown in FIG. 1,system 12 may include electronic device 10 and external electronicdevices 14. Electronic devices in system 12 such as devices 10 and 14may include devices such as a laptop computer, a computer monitorcontaining an embedded computer, a tablet computer, a cellulartelephone, a media player, or other handheld or portable electronicdevice, a smaller device such as a wristwatch device, a pendant device,a headphone or earpiece device, a device embedded in eyeglasses or otherequipment worn on a user's head, or other wearable or miniature device,a television, a computer display that does not contain an embeddedcomputer, a gaming device, a navigation device, an embedded system suchas a system in which electronic equipment with a display is mounted in akiosk or automobile, equipment that implements the functionality of twoor more of these devices, or other electronic equipment.

Device 10 may include control circuitry 16. Control circuitry 16 mayinclude storage and processing circuitry for supporting the operation ofdevice 10 and/or system 12. The storage and processing circuitry mayinclude storage such as hard disk drive storage, nonvolatile memory(e.g., flash memory or other electrically-programmable-read-only memoryconfigured to form a solid state drive), volatile memory (e.g., staticor dynamic random-access-memory), etc. Processing circuitry in controlcircuitry 16 may be used to control the operation of device 10. Theprocessing circuitry may be based on one or more microprocessors,microcontrollers, digital signal processors, baseband processors, powermanagement units, audio chips, application specific integrated circuits,etc.

To support interactions with external equipment, control circuitry 16may be used in implementing communications protocols. Communicationsprotocols that may be implemented using control circuitry 16 includeinternet protocols, wireless local area network protocols (e.g., IEEE802.11 protocols—sometimes referred to as WiFi®), protocols for othershort-range wireless communications links such as the Bluetooth®protocol, cellular telephone protocols, MIMO protocols, antennadiversity protocols, satellite navigation system protocols, millimeterwave communications protocols, IEEE 802.15.4 ultra-widebandcommunications protocols, etc.

Device 10 may include input-output devices 18. Input-output devices 18may be used to allow data to be supplied to device 10 and to allow datato be provided from device 10 to external devices. Input-output devices18 may include user interface devices, data port devices, and otherinput-output components. For example, input-output devices 18 mayinclude one or more image sensors 20, motion sensors 22, display systems24 (e.g., touch screens or displays without touch sensor capabilities),speakers 26, and touch input components 28.

Input-output devices 18 may also include buttons, joysticks, scrollingwheels, touch pads, key pads, keyboards, microphones, haptic elementssuch as vibrators and actuators, status indicators, light sources, audiojacks and other audio port components, digital data port devices, lightsensors, capacitance sensors, proximity sensors (e.g., a capacitiveproximity sensor and/or an infrared proximity sensor), magnetic sensors,and other sensors and input-output components.

Image sensors 20 (sometimes referred to as cameras 20) may include oneor more visible digital image sensors (visible-light cameras) and/or oneor more infrared digital image sensors (infrared-light cameras). Imagesensors 20 may, if desired, be used to measure distances. For example,an infrared time-of-flight image sensor may be used to measure the timethat it takes for an infrared light pulse to reflect back from objectsin the vicinity of device 10, which may in turn be used to determine thedistance to those objects. Visible imaging systems such as a frontand/or rear facing camera in device 10 may also be used to determine theposition of objects in the environment. For example, control circuitry16 may use image sensors 20 to perform simultaneous localization andmapping (SLAM). SLAM refers to the process of using images to determinethe position of objections in the environment while also constructing arepresentation of the imaged environment. Visual SLAM techniques includedetecting and tracking certain features in images such as edges,textures, room corners, window corners, door corners, faces, sidewalkedges, street edges, building edges, tree trunks, and other prominentfeatures. Control circuitry 16 may rely entirely upon image sensors 20to perform simultaneous localization and mapping, or control circuitry16 may synthesize image data with range data from one or more distancesensors (e.g., light-based proximity sensors), with motion data frommotion sensors, and/or other data from other sensors. If desired,control circuitry 16 may use display system 24 to display a visualrepresentation of the mapped environment.

Motion sensors 22 may include accelerometers, gyroscopes, magneticsensors (e.g., compasses), and other sensor structures. Sensors 22 ofFIG. 1 may, for example, include one or more microelectromechanicalsystems (MEMS) sensors (e.g., accelerometers, gyroscopes, microphones,force sensors, pressure sensors, capacitive sensors, or any othersuitable type of sensor formed using microelectromechanical systemstechnology).

Motion sensors 22 may include circuitry for detecting movement andorientation of device 10. Motion sensors that may be used in sensors 22include accelerometers (e.g., accelerometers that measure accelerationalong one, two, or three axes), gyroscopes, compasses, pressure sensors,other suitable types of motion sensors, etc. Storage and processingcircuitry 16 may be used to store and process motion sensor data. Ifdesired, motion sensors, processing circuitry, and storage that formmotion sensor circuitry may form part of a system-on-chip integratedcircuit (as an example).

Display system 24 may be a touch screen display that incorporates alayer of conductive capacitive touch sensor electrodes or other touchsensor components (e.g., resistive touch sensor components, acoustictouch sensor components, force-based touch sensor components,light-based touch sensor components, etc.) or may be a display that isnot touch-sensitive. Capacitive touch screen electrodes may be formedfrom an array of indium tin oxide pads or other transparent conductivestructures.

Display system 24 may include one or more liquid crystal displays,organic light-emitting diode displays, plasma displays, electrophoreticdisplays, microelectromechanical systems displays, electrowettingdisplays, displays with arrays of crystalline semiconductorlight-emitting diode dies, and/or other types of displays.

In some arrangements, display system 24 be a head-mounted display(sometimes referred to as virtual reality glasses or augmented realityglasses). In arrangements where display system 24 is a head-mounteddisplay, the components of device 10 may be contained entirely withinthe head-mounted device or some of the components may be head-mountedand some of the components may be located in a housing that is nothead-mounted. For example, a handheld electronic device such as acellular telephone or tablet computer may serve as an input-outputdevice for a head-mounted display.

In other scenarios, display system 24 may be mounted in a cellulartelephone, tablet computer, watch, or other support structure that isnot attached to a user's head. Arrangements in which display system 24is mounted in a housing that is not head-mounted are sometimes describedherein as an illustrative example.

During operation, images may be displayed for a user on an array ofpixels in display system 24. The images may include computer-generatedimages (e.g., images produced by control circuitry 16 of device 10and/or other control circuitry in system 12), real-time images from avideo camera such as camera 20, real-time images from camera 20 on whichcomputer-generated images are overlaid, and/or other visual content.Images that are displayed as they are captured by camera 20 maysometimes be referred to as live camera images, live camera view images,live camera feed images, live video feed images, etc.

Display system 24 may be used to present computer-generated content suchas virtual reality content and mixed reality content to a user. Ifdesired, virtual reality content may be displayed in the absence ofreal-world content. Mixed reality content, which may sometimes bereferred to as augmented reality content, may include computer-generatedimages that are overlaid on real-world images. The real-world images maybe captured by a camera (e.g., a forward-facing camera) and merged withoverlaid computer-generated content, or an optical coupling system maybe used to allow computer-generated content to be overlaid on top ofreal-world images. As an example, a pair of mixed reality glasses orother augmented reality head-mounted display may include a displaydevice that provides images to a user through a beam splitter, prism,holographic coupler, optical combiner, or other optical coupler.

Augmented reality content on display system 24 may also includecomputer-generated images that are overlaid onto the user's view of thereal world itself, rather than being overlaid onto displayed images ofthe real world. With this type of arrangement, display system 24 mayinclude transparent (see-through) optics such as an optical combinerthat allows a user to view real world objects through display system 24.Display system 24 may present computer-generated images such that theuser sees both the computer-generated images on display system 24 andalso the real world through the display system. The computer-generatedimages enhance or augment the user's view of the real world.

Touch input components 28 may include force sensors and/or touchsensors. Touch input components 28 may include conductive capacitivetouch sensor electrodes or other touch sensor components (e.g.,resistive touch sensor components, acoustic touch sensor components,force-based touch sensor components, light-based touch sensorcomponents, strain gauge components, etc.). Capacitive touch sensorelectrodes may be formed from an array of indium tin oxide pads or othertransparent conductive structures. Touch input components 28 may beconfigured to detect the location of touch input on device 10 and, ifdesired, to measure the amount of force associated with touch input ondevice 10. Touch input components 28 may include touch sensors and forcesensors that work independently of one another (e.g., capacitiveelectrodes that detect touch and one or more strain gauges that detectforce) and/or may include touch sensors that are integrated with forcesensors (e.g., a single sensor may be used to detect touch and force).

Touch input components 28 may include mechanical buttons and/ornon-mechanical buttons. Mechanical buttons may include a mechanicalswitch that is actuated by a button member when the button member isdepressed by a user. Non-mechanical buttons may be formed from solidstate semiconductor materials and/or may include touch sensors such ascapacitive touch sensor electrodes. Non-mechanical buttons do not relyon mechanical switches and therefore can be operated without movement.This is, however, merely illustrative. If desired, non-mechanicalbuttons may be formed from touch sensors on a movable structure (e.g., astructure that moves relative to the housing of device 10 just as amechanical button would move) and/or may be formed from touch sensors ona structure that appears to move without actually moving (e.g., byproviding haptic output that mimics a button press).

Other sensors that may be included in input-output devices 18 includeambient light sensors for gathering information on ambient light levels,proximity sensor components (e.g., light-based proximity sensors,capacitive proximity sensors, and/or proximity sensors based on otherstructures), depth sensors (e.g., structured light depth sensors thatemit beams of light in a grid, a random dot array, or other pattern, andthat have image sensors that generate depth maps based on the resultingspots of light produced on target objects), sensors that gatherthree-dimensional depth information using a pair of stereoscopic imagesensors, lidar (light detection and ranging) sensors, radar sensors, andother suitable sensors.

Control circuitry 16 may use communications circuitry 30 to transmitsignals to and/or receive signals from external equipment such asexternal electronic devices 14. Communications circuitry 30 may includewireless communication circuitry such as one or more antennas such asantenna 32 and associated radio-frequency transceiver circuitry 34.Transceiver circuitry 34 may include wireless local area networktransceiver circuitry (e.g., WiFi® circuitry), Bluetooth® circuitry,cellular telephone transceiver circuitry, ultra-wideband communicationstransceiver circuitry, millimeter wave transceiver circuitry, near-fieldcommunications circuitry, satellite navigation system circuitry such asGlobal Positioning System (GPS) receiver circuitry (e.g., for receivingGPS signals at 1575 MHz or for handling other satellite positioningdata), and/or wireless circuitry that transmits and/or receives signalsusing light (e.g., with light-emitting diodes, lasers, or other lightsources and corresponding light detectors such as photodetectors).Antennas 32 may include monopole antennas, dipole antennas, patchantennas, inverted-F antennas, loop antennas, slot antennas, otherantennas, and/or antennas that include antenna resonating elements ofmore than one type (e.g., hybrid slot-inverted-F antennas, etc.).Antennas 32 may be formed from metal traces on printed circuits or othersubstrates, may include stamped metal parts, may include metalstructures that form part of an enclosure or other supporting structurefor device 10, may include wires and other conductive strands ofmaterial in fabric, and/or other conductive structures.

Device 10 may use communications circuitry 30 to communicate directlywith external device 14, to communicate with a server, and/or tocommunicate with other devices in system 12.

Electronic device 14 of system 12 may include control circuitry 36.Control circuitry 36 may include storage and processing circuitry forsupporting the operation of device 14 and/or system 12. The storage andprocessing circuitry may include storage such as nonvolatile memory(e.g., flash memory or other electrically-programmable-read-only memoryconfigured to form a solid state drive), volatile memory (e.g., staticor dynamic random-access-memory), etc. Processing circuitry in controlcircuitry 36 may be used to gather input from sensors and other inputdevices and may be used to control output devices. The processingcircuitry may be based on one or more microprocessors, microcontrollers,digital signal processors, baseband processors and other wirelesscommunications circuits, power management units, audio chips,application specific integrated circuits, etc.

Electronic device 14 may include input-output devices 40. Input-outputdevices 40 may be used in gathering user input, in gathering informationon the environment surrounding device 14, and/or in providing a userwith output. Devices 40 may include sensors 42. Sensors 42 may includeforce sensors (e.g., strain gauges, capacitive force sensors, resistiveforce sensors, etc.), audio sensors such as microphones, touch and/orproximity sensors such as capacitive sensors, optical sensors such asoptical sensors that emit and detect light, ultrasonic sensors, and/orother touch sensors and/or proximity sensors, monochromatic and colorambient light sensors, image sensors, sensors for detecting position,orientation, and/or motion (e.g., accelerometers, magnetic sensors suchas compass sensors, gyroscopes, and/or inertial measurement units thatcontain some or all of these sensors), muscle activity sensors (EMG),radio-frequency sensors, depth sensors (e.g., structured light sensorsand/or depth sensors based on stereo imaging devices), optical sensorssuch as self-mixing sensors and light detection and ranging (lidar)sensors that gather time-of-flight measurements, humidity sensors,moisture sensors, and/or other sensors. In some arrangements, device 14may use sensors 42 and/or other input-output devices 40 to gather userinput (e.g., buttons may be used to gather button press input, touchsensors overlapping displays can be used for gathering user touch screeninput, touch pads may be used in gathering touch input, microphones maybe used for gathering audio input, accelerometers may be used inmonitoring when a finger contacts an input surface and may therefore beused to gather finger press input, etc.).

If desired, input-output devices 40 of device 14 may include otherdevices 44 such as displays (e.g., to display images for a user), statusindicator lights (e.g., a light-emitting diode that serves as a powerindicator, and other light-based output devices), speakers and otheraudio output devices, electromagnets, permanent magnets, structuresformed from magnetic material (e.g., iron bars or other ferromagneticmembers that are attracted to magnets such as electromagnets and/orpermanent magnets), batteries, etc. Device 14 may also include powertransmitting and/or receiving circuits configured to transmit and/orreceive wired and/or wireless power signals.

To support communications between devices 10 and 14 and/or to supportcommunications between equipment in system 12 and other electronicequipment, control circuitry 36 may communicate using communicationscircuitry 38. Circuitry 38 may include antennas, radio-frequencytransceiver circuitry, and other wireless communications circuitryand/or wired communications circuitry. Circuitry 38 may, for example,support bidirectional wireless communications between devices 10 and 14over wireless link 46 (e.g., a Bluetooth® link, a WiFi® link, a 60 GHzlink or other millimeter wave link, an ultra-wideband communicationslink, a near-field communications link, other suitable wired or wirelesscommunications link, etc.). Device 14 may also include power circuitsfor transmitting and/or receiving wired and/or wireless power and mayinclude batteries. In configurations in which wireless power transfer issupported between devices 10 and 14, in-band wireless communications maybe supported using inductive power transfer coils (as an example).

Wireless signals 46 may be used to convey information such as locationand orientation information. For example, control circuitry 16 in device10 may determine the location of device 14 using wireless signals 46and/or control circuitry 36 in device 14 may determine the location ofdevice 10 using wireless signals 46.

Device 10 may track the location of device 14 using signal strengthmeasurement schemes (e.g., measuring the signal strength of radiosignals from device 10) or using time based measurement schemes such astime of flight measurement techniques, time difference of arrivalmeasurement techniques, angle of arrival measurement techniques,triangulation methods, time-of-flight methods, using a crowdsourcedlocation database, and other suitable measurement techniques. This typeof location tracking may be achieved using ultra-wideband signals,Bluetooth® signals, WiFi® signals, millimeter wave signals, or othersuitable signals. This is merely illustrative, however. If desired,control circuitry 16 of device 10 may determine the location of device14 using Global Positioning System receiver circuitry, proximity sensors(e.g., infrared proximity sensors or other proximity sensors), cameras20, depth sensors (e.g., structured light depth sensors that emit beamsof light in a grid, a random dot array, or other pattern, and that haveimage sensors that generate depth maps based on the resulting spots oflight produced on target objects), sensors that gather three-dimensionaldepth information using a pair of stereoscopic image sensors, lidar(light detection and ranging) sensors, radar sensors, using image datafrom a camera, using motion sensor data, and/or using other circuitry indevice 10.

If desired, angle of arrival measurement techniques may be employed bycontrol circuitry 16 of device 10 and/or control circuitry 36 of device14 to determine the relative orientation of device 10 and device 14. Forexample, control circuitry 16 may determine the orientation of device 10relative to device 14 by determining a phase difference associated withsignals 46 received by antennas 32 in device 10. The phase differencemay be used to determine an angle of arrival of signals 46 received bydevice 10. Similarly, control circuitry 36 of device 14 may determinethe orientation of device 14 relative to device 10 by determining aphase difference associated with signals 46 received by antennas indevice 14. The phase difference may be used to determine an angle ofarrival of signals 46 received by device 14. Angle of arrivalinformation may in in turn be used to determine the direction in which alongitudinal axis of device 10 is being pointed relative to device 14.

FIG. 2 is a perspective view of an illustrative system of the type shownin FIG. 1. System 12 may include electronic device 10 and externalelectronic devices 14. In the example of FIG. 2, external electronicdevices 14 include a tablet computer, a computer monitor, a keyboard, alaptop, a mouse, and a pair of headphones. This is merely illustrative,however. In general, system 12 may include any suitable number and typeof electronic devices.

A user of device 10 may wish to control, gather information from, orotherwise communicate with one or more of external electronic devices 14in system 12. For example, a user of device 10 may wish to wirelesslypair first and second electronic devices 14 (e.g., to wirelessly pair akeyboard or mouse with a computer monitor or laptop, to wirelessly pairheadphones with a tablet computer, to wirelessly stream or share audio,video, device screens, photos, or other data from one device 14 toanother device 14, etc.). A user of device 10 may wish to easily obtaininformation about devices 14 (e.g., current operating state, batterylife, data transfer rate, status of wireless or wired connections withother devices 14, or other information). A user of device 10 may wish tochange a setting or operational state of one or more external devices 14(e.g., to turn device 14 on or off, to adjust the brightness of adisplay or other light source in device 14, to adjust the volume of aspeaker in device 14, etc.).

To provide an intuitive way for a user of electronic device 10 tocontrol, gather information from, or otherwise communicate with externalelectronic devices 14, control circuitry 16 may identify and gatherinformation about external electronic devices in the surroundingenvironment and may present computer-generated images on display system24. The computer-generated images may be overlaid onto images of thereal world (e.g., captured by camera 20 and displayed in real time ondisplay system 24), may be overlaid onto the real world itself (e.g.,the real world as viewed through display system 24 in arrangements wheredisplay system 24 includes an optical combiner), or may be overlaid ontoan entirely virtual world (e.g., an entirely computer-generated image ondisplay system 24).

As shown in FIG. 2, for example, a user of device 10 may operate device10 in an environment. External electronic devices 14 may be located inthe environment. Control circuitry 16 may present the environment ondisplay system 24. The environment presented on display system 24 may beimages of the environment captured by camera 20, may be the actualenvironment viewed through display system 24 (e.g., in arrangementswhere display system 24 has see-through optics such as an opticalcombiner that combines the real world scene with computer-generatedimage light), or may be a virtual environment that is constructed bycontrol circuitry 16 based on information gathered about theenvironment.

The environment presented on display system 24 may include devices 14′.Devices 14′ on display system 24 may be captured images of devices 14,may be actual real world devices 14 viewed through display system 24, ormay be virtual representations of devices 14. The locations of devices14′ on display system 24 may correspond to the locations of devices 14in the real world (e.g., if device A is located to the left of device Bin the real world, device A may be presented on display system 24 to theleft of device B). Devices 14′ on display system 24 may include onlydevices 14 that are in the user's field of view, may include onlydevices 14 that are in the field of view of camera 20, and/or mayinclude all devices 14 within a given environment, whether or not thedevices 14 are viewable or obstructed from view.

A user of device 10 may control or communicate with external devices 14by interacting with devices 14′ on display system 24, by providing othersuitable user input to device 10, and/or by providing input to anothersuitable input device.

To allow a user to intuitively control or communicate with externalelectronic devices 14, control circuitry 16 in device 10 may identifyexternal electronic devices 14. This may include, for example,determining product type (e.g., laptop, computer monitor, keyboard,mouse, headphones, tablet computer, watch, etc.) and gathering otherinformation about the product (e.g., settings, battery life, connectioncapabilities, network capabilities, current wired connections, currentwireless connections, etc.).

Control circuitry 16 may identify and gather information from externalelectronic devices 14 using input-output devices 18 and/orcommunications circuitry 30. For example, control circuitry 16 mayidentify and gather information from external electronic devices 14using data from camera 20 or other light sensor (e.g., by employingimage recognition techniques to identify changes in contrast on device14, colored or textured logos on device 14, corners or edges of device14, product color, logo size relative to display or device size, etc.,by reading a bar code on a display associated with device 14, by readinga bar code on a housing of device 14, by comparing a newly capturedimage of device 14 with a previously captured image of device 14, byreading a code associated with infrared pixels in a display associatedwith device 14, by determining pixel spacing and skew in a displayassociated with device 14, and/or by employing other light-based orimage-based recognition techniques), using depth sensors (e.g.,structured light depth sensors that emit beams of light in a grid, arandom dot array, or other pattern, and that have image sensors thatgenerate depth maps based on the resulting spots of light produced ontarget objects), using sensors that gather three-dimensional depthinformation using a pair of stereoscopic image sensors, using radar orlidar (light detection and ranging) sensors, using radio waves (e.g.,Bluetooth® signals, WiFi® signals, 60 GHz signals or other millimeterwave signals, ultra-wideband communications signals, near-fieldcommunications signals, other suitable wireless communications signals)received by antenna 32 (e.g., by employing signal strength measurementtechniques, phase difference measurement techniques, time of flightmeasurement techniques, time difference of arrival measurementtechniques, angle of arrival measurement techniques, triangulationmethods, etc.), using audio signals received by speaker 26 (e.g., byexchanging ultrasonic signals with device 14 that are at a frequency inor out of a human's audible range), using motion sensor data gathered bymotion sensor 22 (e.g., using an accelerometer, gyroscope, magnetometer,and/or a Global Positioning System receiver to gather information on thelocation and movement of device 10 relative to devices 14, which may beused to supplement image data from camera 20, if desired), and/or usingother data gathered by input-output devices 18 and/or communicationscircuitry 30. Some techniques, such as those that reply upon wirelesscommunications signals, may be used to determine the locations and typesof external electronic devices in the environment even if the externalelectronic devices are obstructed from view (e.g., a router behind awall or other device not in direct line-of-sight of device 10).

Control circuitry 16 may process the data gathered by input-outputdevices 18 to identify a product type of each external electronic device14, to determine the status of wireless communications links betweenexternal electronic devices 14 (e.g., to determine whether such wirelesscommunications links are present and, if present, what types of wirelesscommunications links are present), and/or to determine other informationabout external electronic devices 14.

After control circuitry 16 has identified and gathered the desiredinformation from external devices 14, control circuitry 16 may assigninput functions to respective input regions on display system 24. Theinput regions may be at any suitable location on display system 24. Whena user provides touch input to an input region on display system 24, theinput function associated with that input region may be executed bycontrol circuitry 16. For example, each device 14′ may serve as an inputregion on display system 24 with an associated input function. Controlcircuitry 16 may take appropriate action in response to receiving touchinput on one of devices 14′ (e.g., may display a menu of options ondisplay system 24 for controlling or communicating with device 14, mayturn the device 14 on or off, may adjust settings of device 14, mayestablish or break a wireless connection with device 14, and/or may takeany other suitable action). Establishing a wireless communications linkmay include initiating a new wireless communications link betweendevices and/or activating an existing wireless communications link.Breaking a wireless communications link may include terminating ordeactivating a wireless communications link between devices. This is,however, merely illustrative. If desired, any other suitable portion ofdisplay system 24 may serve as an input region having one or moredesignated input functions for controlling or communicating with devices14. The input regions may be pinned to a particular location in acaptured image (e.g., may be pinned to devices 14′), may be pinned to aparticular location on display system 24, and/or may not be pinned toany particular location on the captured image or display system 24.

Arrangements in which touch input on display system 24 is used tointerface with external devices 14 are merely illustrative. If desired,other types of input such as gesture input from a user's hands may beused to control, communicate with, or receive information about externaldevices 14.

Gesture input may be detected using one or more sensors in device 10(e.g., infrared light cameras, visible light cameras, depth sensors suchas structured light depth sensors, sensors that gather three-dimensionaldepth information using a pair of stereoscopic image sensors, lidar(light detection and ranging) sensors, radar sensors, etc.). If desired,gesture input may be gathered using sensors that are coupled to theuser's hand, fingers, or wrist (e.g., using muscle sensors, motionsensors, optical sensors, or other suitable sensors in a watch, glove,or finger-mounted device to track movement of the user's hand and/orfingers).

Gestures that may be used to control, communicate with, or receiveinformation about external devices 14 include swiping, waving, pinching,pointing, or any other suitable gesture. For example, if a user wishesto switch a pair of earphones to play audio from a laptop instead of acellular telephone, the user may pinch his or her fingers near theearphones (e.g., to mimic grabbing a cord) and may drag his or herfingers to the laptop (e.g., to mimic plugging the cord into thelaptop). Control circuitry 16 may be configured to detect this gestureinput using sensors and may send corresponding control signals to theearphones, laptop, and/or cellular telephone to disconnect the earphonesfrom the cellular telephone and wirelessly connect the earphones to thelaptop. When the user is finished listening to audio from the laptop andwishes to switch back to playing audio from the cellular telephone, theuser may chop, pinch, or provide other gesture input with his or herfingers near the laptop to mimic unplugging the cord from the laptop. Inresponse to this gesture input, control circuitry 16 may send controlsignals to the earphones, cellular telephone, and/or laptop todisconnect the two devices (e.g., so that the audio stops playing fromthe laptop and, if desired, starts playing from the cellular phoneagain). This example is merely illustrative. In general, any suitabletype of gesture input may be used to communicate with, control, orreceive information about external devices 14.

In some scenarios, devices 14′ may appear on display system 24 withoutany overlaid computer-generated content (e.g., display system 24 maypresent only the images captured by camera 20 or display system 24 mayhave see-through optics so that only real world objects are viewed ondisplay system 24). In other scenarios, control circuitry 16 may overlaycomputer-generated images such as computer-generated display elements 48onto display system 24 (e.g., onto the camera-captured images, onto thereal world viewed through display system 24 in arrangements wheredisplay system 24 has see-through optics such as an optical combiner, oronto a virtual reality representation of the environment).Computer-generated display elements 48 may include lines, shapes,arrows, symbols, letters, numbers, characters, colors, on-screen menus,battery charge indicators, status indicators, virtual buttons, icons,sliders, or any other suitable element on display system 24.Computer-generated display elements 48 may be used to convey statusinformation (e.g., battery level, operating temperature, download statusinformation, data transfer rates, network information, volume,brightness, wireless pairing information, or any other suitableinformation), may be used to help inform the user of how to control orcommunicate with devices 14, may be used as input regions on displaysystem 24 with corresponding input functions, may be used to inform theuser when an action has been initiated or completed, may be used toinform the user of a type of connection between devices 14 (e.g.,Bluetooth®, WiFi®, etc.), and or may be used to convey other informationto a user of device 10.

In the example of FIG. 2, computer-generated display elements 48 includearrows indicating which devices 14 are wirelessly communicating witheach other (e.g., which devices are connected over a wireless Bluetooth®or WiFi® communications link), battery status indicators indicating acurrent battery level of one of devices 14, and a symbol such as anexclamation point indicating a message, warning, or other informationassociated with one of devices 14.

Computer-generated display elements 48 may be pinned to a particularobject in the environment. When a computer-generated display element 48is pinned to an object in the environment, its location remains fixedrelative to that object. For example, display elements 48 may include abattery status indicator pinned to monitor 14′ on display system 24 toindicate a battery level of monitor 14. If a user moves device 10,monitor 14′ and the associated display element 48 will move togetheracross display system 24 as the position of device 10 changes. As device10 is pointed towards or brought within the vicinity of new devices 14,those devices 14 will appear on display system 24 as devices 14′, withnew computer-generated display elements 48 specific to those devices 14.

In some arrangements, computer-generated display elements 48 may be usedpurely for providing output to a user of device 10. In otherarrangements, computer-generated display elements 48 may be used forreceiving input from a user of device 10. After control circuitry 16identifies and gathers information from devices 14, control circuitry 16may assign corresponding user input functions to computer-generateddisplay elements 48 and may overlay the computer-generated displayelements 48 onto display system 24 (e.g., onto live camera feed imagesbeing displayed on display system 24, onto the real world viewed throughdisplay system 24 in arrangements where display system 24 hassee-through optics such as an optical combiner, or onto a virtualreality representation of the environment). The user may provide inputto computer-generated display elements 48 and/or devices 14′ on displaysystem 24 by touching display system 24 (e.g., in arrangements wheredisplay system 24 is a touch-sensitive display) and/or by makinggestures (e.g., using his or her hand to touch or otherwise interactwith the user's environment and the computer-generated display elements48 that are virtually overlaid onto the user's environment).

In response to touch input or gesture input to computer-generateddisplay elements 48, control circuitry 16 may take appropriate action.Illustrative input functions that may be performed by control circuitry16 in response to touch input or gesture input on computer-generateddisplay elements 48, touch input or gesture input to devices 14′ ondisplay system 24, touch input or gesture input to other portions ofdisplay system 24, and/or any other suitable user input (e.g., motioninput, audio input, touch or press input on other portions of device 10,etc.) include initiating or establishing a wireless communications linkbetween devices 14, breaking a wireless communications link betweendevices 14, streaming or sharing content (e.g., audio, video, photos, adisplay screen, or other data) from one device (e.g., device 10 or 14)to another device (e.g., device 10 or 14), changing an output, operatingstate, or setting in devices 14 (e.g., turning device 14 on or off,adjusting brightness of a display or other light source in device 14,adjusting the volume of a speaker in device 14, adjusting media playbackoperations on device 14 such as pausing media, changing tracks, etc.,and/or changing other settings or output from device 14), displaying amenu of options or other text on display system 24, and/or performingany other suitable function related to device 10 and/or devices 14.

The use of display system 24 in device 10 is merely illustrative. Ifdesired, a user may control, communicate with, or receive informationabout external devices 14 without using display system 24. Device 10 mayprovide alternative output to the user (e.g., audio output, hapticoutput, other visual output not from display system 24, etc.) or may notprovide any output to the user. For example, a user may provide touchinput or gesture input to device 10 to control or communicate withexternal devices 14 and need not receive any output from device 10 inreturn.

In some arrangements, control circuitry 16 may take action with respectto external devices 14 without requiring user input. Input-outputdevices 18 may include sensors and control circuitry may take actionautomatically based on data gathered using the sensors. For example,eye-tracking sensors in a head-up display may be used to determine whichexternal device 14 is within the user's gaze, and control circuitry 16may take appropriate action depending on which device 14 is in theuser's gaze. If the eye-tracking sensor determines that a user islooking at a computer monitor, control circuitry 16 may establish awireless communications link between a keyboard and the computermonitor. If the eye-tracking sensor determines that the user's gazemoves to a tablet computer display, control circuitry 16 may establish awireless communications link between the keyboard and the tabletcomputer. This allows the user to intuitively control which devicereceives the user's input to the keyboard. If desired, the eye-trackingsensor data may be used to determine which application on a display isbeing viewed (e.g., e-mail application, word processing application,messenger application, web browsing application, etc.), so that typinginput or other user input is directed to the appropriate application.

FIGS. 3, 4, 5, 6, and 7 show illustrative ways in which a user maycontrol, gather information about, or otherwise communicate withexternal electronic devices 14 using device 10.

In the example of FIG. 3, control circuitry 16 of device 10 presents theuser's environment on display system 24. The user's environment ondisplay system 24 may be a live video feed of the user's environment(e.g., captured by camera 20), may be an actual view of the real worldenvironment (e.g., viewed through display system 24 in arrangementswhere display system 24 has see-through optics such as an opticalcombiner), or may be a completely virtual representation of the user'senvironment (e.g., not based on actual camera images). Externalelectronic devices 14 in the environment are presented on display system24 as devices 14′. Devices 14′ may, for example, include a wirelesskeyboard and a computer monitor. Control circuitry 16 of device 10 mayidentify which devices 14 are in the environment and may assigncorresponding user input functions to portions of display system 24. Afirst user input region may be pinned to monitor 14′, a second userinput region may be pinned to keyboard 14′, and additional user inputregions may be located in other portions of display system 24.

If a user wishes to establish a wireless communications link between thekeyboard and computer monitor, for example, the user may swipe his orher finger 56 in direction 50 on display system 24 from keyboard 14′ tomonitor 14′. If the user is providing gesture input instead of or inaddition to touch input, the user may swipe his or her finger in the air(e.g., pointing towards keyboard 14′ initially and moving his or herfinger towards monitor 14′). Control circuitry 16 may process this touchinput and/or gesture input and may send corresponding control signals toone or more of the external devices 14 (e.g., to the keyboard and/or tothe computer monitor) to establish a wireless connection (e.g., aBluetooth® link, a WiFi® link, a 60 GHz link or other millimeter wavelink, an ultra-wideband communications link, a near-field communicationslink, other suitable wired or wireless communications link, etc.)between the external devices 14.

If desired, control circuitry 16 may display text on display system 24in response to touch input to certain regions of an image or gestureinput to certain regions of the environment. For example, if a user'sfinger 56 taps, presses, or otherwise provides touch input on keyboard14′, text 66 may appear on display system 24. Text 66 may provideinformation about keyboard 14 (e.g., battery life, status ofconnections, capabilities, settings, etc.) and/or may provide a userwith options for controlling keyboard 14 (e.g., turning keyboard 14 on,establishing or breaking a wireless connection between the keyboard andthe computer monitor, adjusting settings on keyboard 14, etc.).Similarly, if a user's finger 56 taps, presses, or otherwise providestouch input to monitor 14′ text 66 may appear providing informationabout monitor 14 and/or providing options for controlling monitor 14. Ingeneral, text 66 may appear in response to any suitable touch input ondisplay system 24 and may include information or options for interactingwith any one of external devices 14 in the user's environment. The useof text is merely illustrative. If desired, images without text may begenerated on display system 24 in response to touch input on devices 14′or other portions of display system 24.

FIG. 4 illustrates an example in which a user breaks a connectionbetween external electronic devices 14. Control circuitry 16 of device10 presents the user's environment on display system 24. The user'senvironment on display system 24 may be a live video feed of the user'senvironment (e.g., captured by camera 20), may be an actual view of thereal world environment (e.g., viewed through display system 24 inarrangements where display system 24 has see-through optics such as anoptical combiner), or may be a completely virtual representation of theuser's environment (e.g., not based on actual camera images). Externalelectronic devices 14 in the environment are presented on display system24 as devices 14′.

Control circuitry 16 may identify the type of electronic devices 14 andmay gather other information from electronic devices 14. Controlcircuitry 16 may, for example, receive wireless signals from one or moreof devices 14 indicating that one device 14 is wirelessly paired withanother device 14. For example, a laptop may be sharing its displayscreen or other data with a computer monitor. In response to determiningthat a wireless communications link exists between the laptop andcomputer monitor, control circuitry 16 may overlay computer-generateddisplay element 48 (e.g., an arrow or other element) onto theenvironment on display system 24. Computer-generated display element 48may indicate that content from one device 14 is being shared with orstreamed to another device 14.

If a user wishes to break the wireless communications link between thelaptop and the computer monitor, the user may provide touch input todisplay system 24 and/or may provide gesture input in the air. Forexample, a user may swipe his or her finger 56 in direction 52 ondisplay system 24 or in the air across display element 48. Controlcircuitry 16 may process this touch input or gesture input and may sendcorresponding control signals to one or more of the external devices 14(e.g., to the laptop and/or to the computer monitor) to break thewireless connection (e.g., a Bluetooth® link, a WiFi® link, a 60 GHzlink or other millimeter wave link, an ultra-wideband communicationslink, a near-field communications link, other suitable wired or wirelesscommunications link, etc.) between the external devices 14. This exampleis merely illustrative, however. If desired, other user input to device10 (e.g., other suitable touch input, voice input, motion input, etc.)may be used to break a wireless communications link between externaldevices 14.

FIG. 5 shows an example in which different types of computer-generateddisplay elements 48 are used to convey different types of information toa user. For example, computer-generated display elements 48 overlaidonto display system 24 may include computer-generated display element48-1 and computer-generated display element 48-2. Computer-generateddisplay elements 48-1 and 48-2 may have different visual characteristics(e.g., may have different colors, shapes, sizes, patterns, etc.) toconvey different types of information to a user of device 10.

Computer-generated display element 48-1 with a first visualcharacteristic may indicate a first type of wireless communications link(e.g., a Bluetooth® link, a WiFi® link, a 60 GHz link or othermillimeter wave link, an ultra-wideband communications link, anear-field communications link, other suitable wired or wirelesscommunications link, etc.) between external devices 14, andcomputer-generated display element 48-2 with a second visualcharacteristic different from the first visual characteristic mayindicate a second type of wireless communications link different fromthe first type of wireless communications link. For example, a blue linebetween two devices 14′ may indicate that the two devices 14 arewirelessly paired over a Bluetooth® communications link, whereas a redline between two devices 14′ may indicate that the two devices 14 arewirelessly paired over a WiFi® communications link. This is merelyillustrative, however. In general, any suitable characteristic ofcomputer-generated display elements 48 may be varied to convey anysuitable type of information about external devices 14 and/or the typesof options a user may have to interact with external devices 14.

FIG. 6 shows an example in which user input on computer-generateddisplay elements 48 causes text 54 to be displayed on display system 24.Text 54 may be a menu of options from which a user may select suitableactions to take with respect to external devices 14 (e.g., turningdevices 14 on or off, establishing or breaking a wireless connectionbetween external devices 14, adjusting settings on external devices 14,etc.) and/or may provide information on external devices 14 (e.g.,battery life, status of connections, capabilities, settings, etc.).

FIG. 7 shows an illustrative example in which a user controls externaldevice 14 using device 10. Control circuitry 16 of device 10 presentsthe user's environment on display system 24. The user's environment ondisplay system 24 may be a live video feed of the user's environment(e.g., captured by camera 20), may be an actual view of the real worldenvironment (e.g., viewed through display system 24 in arrangementswhere display system 24 has see-through optics such as an opticalcombiner), or may be a completely virtual representation of the user'senvironment (e.g., not based on actual camera images). In the example ofFIG. 7, external electronic device 14 in the user's environment is alamp or other light source having an adjustable brightness. Controlcircuitry 16 may identify external device 14 and may assigncorresponding user input functions to display system 24 based on theidentified external device 14. For example, a user may swipe up indirection 62 on display system 24 to increase the brightness of lightfrom external device 14 and may swipe down in direction 64 to decreasethe brightness of light from external device 14.

In arrangements where multiple external electronic devices are in theuser's environment (and presented on display system 24), different typesof touch or gesture input may correspond to different types of controlsignals for the external electronic devices. For example, a swipe withone finger may control a single external electronic device 14 (e.g., mayadjust the brightness of the light source towards which device 10 ispointed), a swipe with two fingers may control two external electronicdevices 14 (e.g., may adjust the brightness of two light sources in theuser's vicinity), and a swipe with three fingers may control all of theexternal electronic devices 14 in the user's vicinity (e.g., may adjustthe brightness of all of the light sources in the same room). As anotherexample, a single finger swipe from one device 14′ to another device 14′may result in a first user input function (e.g., setting the two devicesinto dual screen mode), whereas a double finger swipe from one device14′ to another device 14′ may result in a second user input function(e.g., setting the two devices into screen share mode).

FIGS. 8, 9, and 10 show illustrative ways in which external electronicdevices 14 may be identified by device 10.

As shown in FIG. 8, external electronic devices 14 may include one ormore displays such as display 58 and one or more logos such as logos 68.Device 10 may use input-output devices 18 such as camera 20 or otherlight sensor to pick up visual features on external devices 14. Controlcircuitry 16 may process the captured images or other light sensor datato identify the type of device and/or gather other information aboutdevices 14. For example, camera 20 may capture images of externaldevices 14, and control circuitry 16 may process the captured images andperform image recognition techniques to identify devices 14 (e.g., byidentifying logo 68 on device 14, changes in contrast on device 14,colored or textured logos on device 14, corners or edges of device 14,product color, logo size relative to display or device size, etc.). Alight source such as one or more infrared or visible light-emittingdiodes on external electronic device 14 may flash on and off to transmita binary recognition code that is captured by camera 20 or other lightsource in device 10 and read by control circuitry 16 to identifyexternal electronic device 14. Pixels in display 58 of device 14 such ascorner pixels 70 may be infrared pixels that emit light corresponding toa recognition code that is picked up by camera 20 and read by controlcircuitry 16. Control circuitry 16 may be configured to determine pixelspacing and skew in display 58 of device 14 based on images from camera20, which may in turn be used to identify device 14. If desired, device14 may upload an image of what is currently being displayed on display58 (e.g., an identical image or a reduced-resolution image of what isbeing displayed on display 58) to a server and/or may send the image todevice 10, and control circuitry 16 may analyze images from camera 20 todetermine if a matching image is located in the captured images.

If desired, control circuitry 16 may determine if external devices 14are facing towards device 10 or away from device 10. For example, logos68 may appear differently on the back of device 14 than on the front ofdevice 14, which may be detected by camera 20. Control circuitry 16 maytake suitable action in response to determining that an external device14 is faced away from device 10. This my include, for example,overlaying computer-generated display element 48 onto the device 14′ ondisplay system 24 to indicate that the device 14 is not available and/orto indicate that device 10 does not have access to that device 14. Thecomputer-generated display element 48 may be a shade of grey overlappinga portion of the device 14′ or may be any other suitable displayelement.

In the example of FIG. 9, visual markers such as visual markers 74 arelocated on external devices 14. Visual markers 74 may be ultravioletink, visible ink, infrared ink, physical marks, or other suitable visualmarkers that may be captured by camera 20 and read by control circuitry16 to identify external devices 14.

In the example of FIG. 10, display 58 of external electronic device 14displays a bar code 72 that is captured by camera 20 and read by controlcircuitry 16 to identify external electronic device 14. The use of a barcode is merely illustrative, however. If desired, display 58 may displayother objects, patterns, etc., which may in turn be captured by camera20 and read by control circuitry 16.

FIG. 11 is a perspective view of an illustrative system in which a userinteracts directly with a pair of external electronic devices toestablish or break a wireless connection between the external electronicdevices. Devices 14 may include touch screen displays 58. A user may, ifdesired, swipe from one display 58 to another display 58 in direction60. Control circuitry 36 in each of the devices 14 may exchange wirelesssignals with the other device 14 using communications circuitry 38 sothat the two devices 14 can compare the touch inputs to each display 58(e.g., to determine the time difference between the touch inputs to eachdisplay 58). If the time difference and/or physical distance between thetwo devices 14 is within a predetermined range, control circuitry 36 ineach device may establish a connection between the two devices 14 (e.g.,a Bluetooth® link, a WiFi® link, a 60 GHz link or other millimeter wavelink, an ultra-wideband communications link, a near-field communicationslink, other suitable wired or wireless communications link, etc.).

The use of touch input on touch screen displays 58 of external devices14 is merely illustrative. If desired, the user may provide gestureinput by moving his or her hand in the air in direction 60. The gestureinput may be detected using sensors in device 10 and/or using sensors inone or both of external devices 14.

The foregoing is merely illustrative and various modifications can bemade to the described embodiments. The foregoing embodiments may beimplemented individually or in any combination.

What is claimed is:
 1. A head-mounted device, comprising: a head-mounteddisplay configured to present images of a real-world environment thatincludes an external electronic device; control circuitry configured tooverlay computer-generated display elements onto the images of thereal-world environment, wherein the computer-generated display elementscomprise an option for controlling an operational setting of theexternal electronic device; and a sensor that gathers gesture input,wherein the control circuitry is configured to send wireless controlsignals for controlling the operational setting to the externalelectronic device in response to the gesture input.
 2. The head-mounteddevice defined in claim 1 wherein the sensor comprises an infrared lightcamera.
 3. The head-mounted device defined in claim 1 wherein the sensorcomprises a visible light camera.
 4. The head-mounted device defined inclaim 1 wherein the operational setting is selected from the groupconsisting of: brightness level, volume level, power status, and mediaplayback status.
 5. The head-mounted device defined in claim 1 whereinthe gesture input comprises gesture input to the computer-generateddisplay elements.
 6. The head-mounted device defined in claim 1 whereinthe gesture input comprises hand gestures in the air.
 7. Thehead-mounted device defined in claim 1 wherein the computer-generateddisplay elements comprise information about the external electronicdevice.
 8. The head-mounted device defined in claim 7 wherein theinformation is selected from the group consisting of: battery life,wireless connection status, and network capabilities.
 9. A head-mounteddevice, comprising: a head-mounted display configured to present imagesof a real-world environment that includes first and second externalelectronic devices; a sensor that gathers gesture input; and controlcircuitry configured to initiate wireless pairing between the first andsecond external electronic devices in response to the gesture input. 10.The head-mounted device defined in claim 9 wherein the gesture inputcomprises a hand swipe between the first and second external electronicdevices.
 11. The head-mounted device defined in claim 9 wherein thecontrol circuitry is configured to overlay computer-generated displayelements onto the images of the real-world environment in response tothe gesture input.
 12. The head-mounted device defined in claim 11wherein the computer-generated display elements include a line extendingbetween the first and second external electronic devices.
 13. Thehead-mounted device defined in claim 12 wherein the sensor gathersadditional gesture input and wherein the control circuitry is configuredto break a wireless communications link between the first and secondexternal electronic devices in response to the additional gesture input.14. The head-mounted device defined in claim 9 wherein the sensorcomprises an infrared light camera.
 15. The head-mounted device definedin claim 9 wherein the sensor comprises an visible light camera.
 16. Ahead-mounted device, comprising a head-mounted display configured topresent images of a real-world environment that includes an externalelectronic device; a sensor that gathers gesture input; and controlcircuitry configured to overlay computer-generated display elements ontothe images of the real-world environment in response to the gestureinput, wherein the computer-generated display elements compriseinformation associated with the external electronic device.
 17. Thehead-mounted device defined in claim 16 wherein the information isselected from the group consisting of: battery level, brightness level,volume level, power status, wireless connection status, and networkingcapabilities.
 18. The head-mounted device defined in claim 16 whereinthe sensor comprises a camera.
 19. The head-mounted device defined inclaim 16 wherein the sensor gathers additional gesture input and whereinthe control circuitry is configured to send wireless control signals tothe external electronic device in response to the additional gestureinput.
 20. The head-mounted device defined in claim 19 wherein wirelesscontrol signals comprise wireless control signals for adjusting anoperational setting of the external electronic device.